Electronic watermarking in the compressed domain utilizing perceptual coding

ABSTRACT

A method and apparatus are described for inserting a watermark in the compressed domain. The watermark inserted does not require a reference. An overall watermarking system incorporating the invention combines cleartext, bitstream, and integrated watermarking. In a perceptual coder, the data enters a filterbank, where it is processed into multiple separate coefficients. A rate/distortion control module uses noise threshold information from a perceptual coder, together with bit-count information from a noiseless coder, to compute scale factors. The coefficients are multiplied by the scale factors and quantized, then noiseless coded and then output for further processing/transmission. The invention supports three embodiments for inserting a mark into the bitstream imperceptibly. It is assumed that some set of scale factor bands have been selected, into which mark data will be inserted. In one embodiment, a set of multipliers {x i =2 Ni : iεM} is chosen. Each triple is modified by dividing the scale factor by x, i  multiplying the quantized coefficients by {x i }, and adding mark data to the non-zero modified quantized coefficients. In an alternate embodiment, watermark data is represented via two characteristics of the bitstream data. A Huffinan table is selected for encoding the Scale Factor Band receiving watermark data which is not the table that would normally be used. The watermark data bit is set according to any desired scheme, and the quantized coefficients are derived using the alternate Huffinan table. In another embodiment, watermarking is integrated with quantization. The watermark is therefore difficult to remove without perceptible effects. The fact that marking data is present is again indicated by characteristics of the bitstream data. The modification factors {x i } are now all close to unity.

RELATED APPLICATIONS

This application claims priority under Title 35, United States CodeSections 199(e) from U.S. Provisional Application Serial No. 60/067,225,filed Dec. 3, 1997.

FIELD OF THE INVENTION

The present invention relates to electronic watermarking of datastreamsand, in particular, to an imperceptible watermark which is inserted inthe compressed domain and can be detected without a reference.

BACKGROUND OF THE INVENTION

Electronic distribution of multimedia content is an important byproductof the confluence of recent technological advances. Increasing networkbandwidth, compression algorithms that preserve audio and video qualitywhile reducing bit rate dramatically, higher density storage devices,and network search engines, when taken together, support networkservices which are revolutionizing the distribution of music and video.

Content owners naturally wish to maintain control over the distributionof their wares. To effectively protect their intellectual property (IP),an integrated system design is necessary [J. Lacy, D. P. Maher, and J.H. Snyder, “Music on the Internet and the Intellectual PropertyProtection Problem”, Proc. International Symposium on IndustrialElectronic, Guimaraes, Portugal, July 1997]. A typical protection systemconsists of three major building blocks. First, compressed content isstored in a cryptographic container before distribution to users.Second, a flexible licensing mechanism is utilized to answer questionsabout the trustworthiness of those seeking access to the content. Third,watermarks are embedded in the content in an imperceptible fashion inorder that the content can be identified if the cryptographic containerhas been breached. A secure system design integrates these threecomponents.

An electronic watermark is a data stream inserted into multimediacontent. It contains information relevant to the ownership or authorizeduse of the content. Watermarks typically serve one of three functions:identification of the origin of the content, tracing of illegallydistributed copies of the content, and disabling of unauthorized accessto the content. No single marking method is best suited to all threefunctions, both because of complexity and because different functionsand marking algorithms are resistant to different kinds of attacks. Anysingle piece of music or video can therefore be expected to be markedwith a variety of different methods.

For copyright identification, every copy of the content can be markedidentically, so the watermark needs to be inserted only once prior todistribution. Ideally, detection should not require a reference, becausea search engine has no apriori way to identify the work from which itmust recover the mark. The watermark particularly needs to be detectableinside an edited work in which the original content may be eithershortened or abutted with other works. Not only must the watermark beshort enough to be detected in a shortened version of the work, but somemeans must be provided to synchronize the detection process in orderthat the watermark can be located in the processed bitstream. Finally, awatermark used for copyright identification must be robust to furtherprocessing. Any attempt to remove it, including re-encoding the content,should lead to perceptible distortion.

Transaction identification requires a distinct mark for eachtransaction. The primary challenge of point-of-sale marking is to movethe content through the watermarking engine quickly, meaning that thealgorithm used must be of low complexity. One strategy that meets thisrequirement is to inert the watermark in the compressed domain. Ideally,mark insertion should increase the data rate very little. In contrast tocopyright ownership marking, the transaction identification watermarkmust be robust to collusion attacks.

Disabling access to content is generally best performed by mechanismsother than watermarks. If a watermark is used to disable access tocontent, the watermark recovery mechanism should be of low complexity.It should not be used as a protection of last resort, however, asdisabling access clearly indicates the location of the watermark toanyone who can reverse-engineer the access mechanism.

Watermarks used in conjunction with compression algorithms fall into oneof three classes: cleartext (PCM) marking, bitstream marking, andmarking integrated with the compression algorithm. Each type hasadvantages and disadvantages. The intended use of the watermark directlyaffects the choice of algorithm.

Cleartext marking relies on perceptual methods to imperceptibly embed adata stream in a signal. The model for many cleartext marking algorithmsis one in which a signal is injected into a noisy communication channel,where the audio/video signal is the interfering noise [J. Smith, B.Comisky, “Modulation and Information Hiding in Images”, Proc. FirstInternational Information Hiding Workshop, LNCS 1174, Springer-Verlag,Cambridge, U.K., May/June 1996, pp. 207-226]. Because the channel is sonoisy and the mark signal must be imperceptible, the maximum bit ratesthat are achievable for audio are generally less than 100 bps.

A cleartext mark appears in all processed generations of the work, sinceby design the marking algorithm is both secure and robust in the face oftypical processing. It is therefore well suited to identification of thework. There are two major disadvantages to cleartext marking. First,because such algorithms compute a perceptual model, they tend to be toocomplex for point-of-sale applications. Second, a potentiallysignificant problem, is that these algorithms are susceptible toadvances in the perceptual models used in compression algorithms. Manycleartext marking algorithms have been reported [see, e.g. Proceedingsof the Fourth International Conference on Image Processing, SantaBarbara Calif., October 1997].

Retrieval mechanisms for cleartext watermarks fall into two classes:reference necessary and reference unnecessary. In either case, themechanism for mark recovery is generally of high complexity.Furthermore, if means for detecting these watermarks are embedded in aplayer, an attacker, by reverse engineering the player, may be able toidentify and remove the marks. Cleartext watermarks typically should notbe used to gate access to content.

Bitstream marking algorithms manipulate the compressed digital bitstreamwithout changing the semantics of the audio or video stream. Forexample, a data envelope in an MPEG-2 Advanced Audio Coding (AAC) [IS13818-7 (MPEG-2 Advanced Audio Coding, AAC), M.Bosi, K. Brandenburg, S.Quackenbush, M. Dietz, J. Johnston, J. Herre, H. Fuchs, Y. Oikawa, K.Akagiri, M. Coleman, M. Iwadare, C. Lueck, U. Gbur, B. Teichmann] audioframe could contain a watermark, albeit one which could easily beremoved. Bitstream marking is low-complexity, so it can be used to carrytransaction information. However these marks cannot survive D/Aconversion and are generally not very robust against attack; forexample, they are susceptible to collusion attacks. Because the marksignal is unrelated to the media signal, the bit rate that thesetechniques can support can be as high as the channel rate. This type ofmark can be easily extracted by clients and is thus appropriate forgating access to content.

Integrating the marking algorithm with the compression algorithm avoidsan ‘arms race’ between marking and compression. Since the perceptualmodel is available from the workings of the compression algorithm,integrated marking algorithms alter the semantics of the audio or videobitstream, thereby providing resistance to collusion attacks. Dependingon the details of the marking algorithm, the mark may survive D/Aconversion. An example of this approach is described by F. Hartung andB. Girod in “Digital Watermarking of MPEG-2 Coded Video inthe BitstreamDomain”, Proc. IEEE ICASSP, pp. 2621-4, April 1997. The method ofHartung and Girod does not use perceptual techniques.

A watermark which can be recovered without a priori knowledge of theidentity of the content could be used by web search mechanisms to flagunauthorized distribution of the content. Since media are compressed onthese sites, a mark detection algorithm that operates in the compresseddomain is useful. Accordingly, it is a primary object of the presentinvention to provide a robust integrated watermark that is inserted intoaudio or video data in the compressed domain utilizing perceptualtechniques.

SUMMARY OF THE INVENTION

This invention integrates watermarking with perceptual codingmechanisms. A first generation technique is described which insertsdata, typically a watermark, into an audio or video bitstreamcooperatively with the compression algorithm. The data may be recoveredwith a simple decoding process. It is robust to attacks which modifybitstream scale factors, in the sense that damaging the mark producesperceptible artifacts. The watermarking technique of the presentinvention can be detected in the compressed domain without a reference,thereby avoiding a complete decode. An overall watermarking systemincorporating the invention combines source (cleartext), bitstream(non-semantic altering), and integrated (semantic altering)watermarking.

In a generic perceptual coder according to the invention, the audio orvideo data enters the filterbank, where it is processed into multipleseparate coefficients. The perceptual model module computes noisethreshold information for the coefficients. The rate/distortion controlmodule uses this information, together with bit-count informationreceived from a noiseless coding module, to compute the scale factors tobe used. For audio data, the scale factors module multiplies thecoefficients received from the filterbank by the scale factors receivedfrom rate/distortion control and sends the resulting quantities to theQuantizer. For video data, the scale factors are used by the Quantizerto quantize the coefficients. For both audio and video data, thequantized coefficients from Quantizer are noiseless coded and then sentto the bitstream multiplexor. The coded data is then output from thebitstream multiplexor for further processing and transmission. Theintegrated marking technique of the present invention is particularlyimplemented by the perceptual modeling, rate/distortion control,quantization, and noiseless coding modules.

In the methods of the present invention, A={f_(i), H_(i), {q_(ij)}} isthe set of triples of scale factors f_(i), Huffman tables H_(i), andquantized coefficients {q_(ij)}. The present invention supports threedifferent embodiments for inserting a mark into the bitstreamimperceptibly. It is assumed in these embodiments that some set of scalefactor bands have been selected, into which mark data will be inserted.The specific method by which SFB are chosen for marking is notspecified; however the marking set will be dynamic. M is the set ofindices associated with the set of SFB chosen for marking.

In one embodiment, a set of multipliers {x_(i)=2^(Ni): iεM} is chosen.Each triple {f_(i), H_(i), {q_(ij)}: iεM} is modified by dividing thescale factor by x_(i), multiplying the quantized value {q_(ij)} by{x_(i)}, and adding mark data {m_(ij)} to the non-zero modifiedquantized values. The Huffman table for the modified SFB is now thesmallest codebook that accommodates the largest valueq_(ij)×x_(i)+m_(ij). Finally, the integrally watermarked encoded sourceis output from the perceptual coder. Since the original scale factorswere chosen perceptually, the resulting mark is imperceptible.

In an alternate embodiment, applicable only to audio, the watermark datais represented via two particular characteristics of the bitstream data.The indication that watermark data is present is that the Huffman tableused to encode the SFB is not the table that would ordinarily be used.The watermark data bit is set according to any desired scheme, and thequantized coefficients are derived using the alternate Huffman table.Finally, the integrally watermarked encoded source is output from theperceptual coder.

Another embodiment is a method for watermarking which is integrated withquantization. The watermark is therefore difficult to remove withoutperceptible effects. The fact that marking data is present is againindicated by characteristics of the bitstream data. The watermark bit(s)are set before quantization. The modification factors {x_(i)} are allnow close to unity. The resulting Huffman table for an SFB thereforewill be the original Huffman table or the next larger codebook. Becausethe modification to the spectral coefficients occurs beforequantization, the changes to the reconstructed coefficients will bebelow the perceptual threshold.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is an overview block diagram of an embodiment of a watermarkingsystem incorporating one embodiment of the present invention;

FIG. 2 is a simplified block diagram of an embodiment of a genericperceptual coder according to the present invention;

FIG. 3 illustrates an embodiment of the method of perceptual codingaccording to the present invention:

FIG. 4 illustrates an alternate embodiment of the method of perceptualcoding according to the present invention;

FIG. 5 illustrates another embodiment of the method of perceptual codingaccording to the present invention;

FIG. 6 is a table displaying the results of an audio simulationutilizing the embodiment of FIG. 5; and

FIG. 7 is a table displaying the results of a video simulation utilizingthe embodiment of FIG. 5.

DETAILED DESCRIPTION

The watermarking technique of the present invention can be detected inthe compressed domain without a reference, thereby avoiding a completedecode. As shown in FIG. 1, an overall watermarking system incorporatingthe invention is a first generation system that combines source,bitstream, and integrated watermarking. In the system of FIG. 1, thesource (cleartext) data 110 is optionally injected 120 with a sourcewatermark 130. Typically, this cleartext (source) watermarking is PCM(Pulse Code Modulation) marking, but any of the many other forms ofcleartext marking known in the art would be suitable.

The source data with its (optional) newly added cleartext watermark isthen passed to the perceptual coder (encoder) 150, where the data isfurther marked with a second watermark 140 via a process that isintegral to the compression process. The invention specificallycomprises these integrated watermarking components 140 and 150 of FIG.1, embodiments of which are described in detail in conjunction withFIGS. 3-5. The integrated watermarking and coding process of theinvention is a semantic altering process.

After the compression and marking process 150, the compressed data isoptionally combined 160 with a third watermark 170 via a bitstreammarking process. Typically, this involves one or more XOR operations,but any bitstream marking technique known in the art would be suitable.The bitstream watermarking process is non-semantic altering. After(optional) bitstream watermarking, the compressed and watermarked datais transmitted or otherwise provided to the output device, where it isdecoded 160 and provided to the user.

The integrated watermarking system of the invention can be configured tosupport the three primary marking functions. As depicted in FIG. 1, itdoes not include, but is compatible with, use of front-end cleartextmarking algorithm. It is assumed that the cleartext original is notavailable to any parties, except possibly auditors seeking to recoverthe watermark. In particular, the cleartext original is not available toattackers. The decompressed and marked content will generally beavailable to everyone.

In MPEG Advanced Audio Coding (AAC), spectral lines are grouped into 49“scale factor bands” (SFB), each band containing between 4 and 32 lines.Associated with each band is a single scale factor, which sets thequantizer step-size, and a single Huffman table (AAC employs 11non-trivial Huffman tables). The coefficient for each spectral line isrepresented by an integer (quantized) value. In MPEG video, a blockconsists of 64 coefficients, and each set (termed a macroblock) of 6blocks has an associated quantization step-size Q_(p). The same Huffmantable is used for the coefficients for all Q_(p) values. As with audio,each coefficient is represented by an integer after quantization.Because the watermarking algorithms for audio and video are essentiallyidentical, for consistency audio terminology (scale factor) is usedherein when techniques are discussed. When the results for video arediscussed in connection with FIGS. 6 and 7, terminology specific tovideo is used.

FIG. 2 is a simplified block diagram of an embodiment of a genericperceptual coder according to the present invention. As seen in FIG. 2,the audio or video data 210 enters the filterbank 220, where it isprocessed into multiple separate coefficients. Information about whattype of filterbank 220 was used is sent to the bitstream multiplexor280. The coefficients themselves are passed to the scale factor module250 and to the perceptual model module 230. The perceptual model module230 computes noise threshold information for the coefficients and passesthis information to the rate/distortion control module 240. Therate/distortion control module 240 uses this information, together withbit-count information received from a noiseless coding module 270, tocompute the scale factors to be used, which are then passed to the scalefactor module 250.

For audio data, the scale factors module 250 multiplies the coefficientsreceived from the filterbank 220 by the scale factors received fromrate/distortion control 240 and sends the resulting quantities to theQuantizer 260. For video data, the scale factors are used by theQuantizer 260 to quantize the coefficients. For both audio and videodata, the quantized coefficients from Quantizer 260 are noiseless coded270 and then sent to the bitstream multiplexor 280. The coded data isthen output 290 from the bitstream multiplexor 280 for furtherprocessing and transmission. The integrated marking technique of thepresent invention is particularly implemented by the perceptual modeling230, rate/distortion control 240, quantization 260, and noiseless coding270 modules of FIG. 2.

In the description of the methods of the present invention, A={f_(i),H_(i), {q_(ij)}} is the set of triples of scale factors f_(i), Huffmantables H_(i), and quantized coefficients {q_(ij)}. Note that only oneHuffman table is used in video. The present invention supports threedifferent embodiments for inserting a mark into the bitstreamimperceptibly. It is assumed in these embodiments that some set of scalefactor bands have been selected, into which mark data will be inserted.The specific method by which SFB are chosen for marking is notspecified; however, for audio, SFB encoded with the null Huffman tableH₀ should probably not be marked. For video, zero coefficients shouldremain zero and not be modified. Hence, the marking set will be dynamic.M is the set of indices associated with the set of SFB chosen formarking.

One embodiment of the method for electronic watermarking in thecompressed domain utilizing perceptual coding is illustrated by the flowdiagram in FIG. 3. As illustrated in FIG. 3, first a set of triples isestablished 310 for each SFB that is to receive watermark data. Next, aset of multipliers {x_(i)=2^(Ni): iεM} is chosen 320. Each triple{f_(i), H_(i), {q_(ij)}: iεM} is modified by dividing 330 the scalefactor by x_(i), multiplying 340 the quantized value {q_(ij)} by{x_(i)}, and adding 350 mark data {m_(ij)} to the non-zero modifiedquantized values. Finally, the integrally watermarked encoded source isoutput 360 from the perceptual coder 150 of FIG. 1. Mathematically, theresult of this perceptual coding step can be represented: A→A′, where

∀i: iεM, {f _(i) ′, H _(i) ′, {q _(ij) ′}}={f _(i) , H _(i) , {q_(ij)}},

∀i: iεM, {f _(i) ′, H _(i) ′, {q _(ij) ′}}={f _(i) /x _(i) , H _(i) ″,{q _(ij) ×x _(i) +m _(ij)}},

where H_(i)″ is the smallest codebook that accommodates the largestvalue q_(ij)×x_(i)+m_(ij).

Since the original scale factors were chosen perceptually, the resultingmark is imperceptible. A feedback mechanism similar to the one describedby Hartung and Girod can be used to prevent modification of scalefactors that would increase the bit rate significantly. It should benoted that if the attacker can identify the frame and SFB containing themark data, then that data can easily be removed. A possible attack onthis method would be to run a perceptual model on the decompressedoutput. While it is unlikely that the perceptual model could indicateunambiguously every marked location, it seems likely that many could beidentified.

An alternate embodiment, applicable only to audio data, is illustratedby the flow diagram in FIG. 4. In this embodiment, the watermark data isrepresented via two particular characteristics of the bitstream data.The indication that watermark data is present is that the Huffman tableused to encode the SFB is not the table that would ordinarily be used.The value of the watermark data bit (one bit per SFB) can be indicatedin many ways; for example, if the SFB index is even, the value is 0,otherwise 1. Mathematically, this is represented: {f_(i), H_(i),{q_(ij)}}→{f_(i), H_(i)′, {q_(ij)}}. As illustrated in FIG. 4, the scalefactor is established 410 for the SFB to receive watermark data. AHuffman table is then selected 420 for encoding SFB that can stillencode all the coefficients with the required dynamic range . Thewatermark data bit is set 430 according to any desired scheme, and thequantized coefficients are derived 440 using the alternate (non-usual)Huffman table. Finally, the integrally watermarked encoded source isoutput 450 from the perceptual coder 150 of FIG. 1.

It should be noted that, in this method, sectioning, a process by whichcodebooks are “promoted” to reduce bit rate, introduces similar changesin the choice of codebooks. That is, sectioning itself can erase themark data indication. Also, this marking is particularly easy toidentify, since an attacker looking at the bitstream can observe thatthe codebook used to encode the coefficients in the SFB is not theminimum codebook required. However, by a sensible choice of SFB, it ispossible to insert mark data in a way that will not be modified bysectioning but rather mimics the action of sectioning and therefore issomewhat less obvious to an attacker.

The methods of FIGS. 3 and 4 are coupled to the encoder 150 of FIG. 1only via the overall bit rate limit.

Another embodiment of the invention, illustrated by the flow diagram inFIG. 5, is a method for watermarking which is fully integrated withquantization. The watermark is therefore difficult to remove withoutperceptible effects. As in the embodiment of FIG. 4, the fact thatmarking data is present is indicated by characteristics of the bitstreamdata. The watermark data bit(s) are set before a quantization step. Asin the embodiment of FIG. 3, the scale factor f_(i) and the normalizedspectral coefficients {q_(ij)} are modified by a factor x_(i), but nowall {x_(i)} are close to unity. The normalized spectral coefficients{q_(ij)} referred to in audio will also be referred to herein as simply“coefficients.” If {v_(ij)} is the set of spectral coefficients prior toquantization, and Q_(i) is the quantizer for SFB i, i.e.∀i{q_(ij)}=Q_(i)[{v_(ij)}], then mathematically:

{f _(i) , H _(i) , {q _(ij) }}→{f _(i) ′, H _(i) ′, {q _(ij)′}},

where

f_(i)′=f_(i)/x_(i)

q_(ij)′=Q_(i)[x_(i)×v_(ij)]

H_(i)′=H_(i) or the next larger codebook appropriate for q_(ij)′; and

x_(i)≅1

Because the modification to the spectral coefficients occurs beforequantization, the changes to the reconstructed coefficients will bebelow the perceptual threshold. If this change were introduced afterquantization, the change in some quantized values would be greater thanthe perceptual noise floor. Equivalently, an attacker who modifies thequantized values to eradicate or modify the mark will be introducingenergy changes that exceed the noise floor. Because the changes instep-sizes will be small, because not all coefficients will change, andbecause the attacker will not have access to the uncompressed cleartextsource material, the attacker will generally not be able to identifythose SFB which are used for marking. Further, the change in bit rateassociated with marking should be small. In this third embodimentmethod, the value of the watermark bit can be indicated in a variety ofways, e.g. it might take on the value of the Least Significant Bit (LSB)of the scale factor value, in which case a scale factor needs to bemodified only if its LSB differs from the desired value. For both audioand video, the increase in bit count incurred by this method must bemonitored.

As illustrated in the flow diagram in FIG. 5, the watermark data bit(s)are set according to any desired scheme in step 510. Then, the scalefactors are established from perceptual thresholds at step 520. With thewatermark bit(s) set and the scale factors established, the next step isto establish a plurality of scale factor bands, M, in which to locatethe set watermark bit(s) at step 530. With the bands established, thenext step is to choose an appropriate set of multipliers {x_(i)≅1: iεM}at step 540. Then, at step 550, each triple {f_(i), H_(i), {q_(ij)}:iεM} is modified by dividing the scale factor by x_(i). This results inmodified set f_(i)′. The normalized spectral coefficients then aremultiplied at step 560 by respective multipliers and quantized resultingin q_(ij)′=Q_(i)[x_(i)×v_(ij)]. Now, in step 570, a Huffman Table H_(i)′is chosen to be appropriate for q_(ij)′. This may be H_(i) or the nextlarger codebook appropriate for q_(ij)′. Finally, the integrallywatermarked encoded source is output from the perceptual coder 150 ofFIG. 1.

Generally watermark sequences are inserted a few bits per frame. Thedata to be carried by the stream is typically mapped into a markingsequence prior to embedding, where the characteristics of the mappingfunction depend on the type of attack expected. Indeed, since there maybe a wide range of attacks, the data may be redundantly mapped indifferent ways in the hope that at least one mapping will survive allattacks. This leads to the issue of recognizing where a marking sequencebegins. One approach is to use synchronizing codes. However the attackermay be able to identify these codes, and if the attacker can eliminateor damage the codes, recovery of mark data may not be possible.

In the system of the present invention, synchronization is tied to frameboundaries. The scale factors included at the beginning of the frame aremodified by modifying the LSBs so that they represent a sequence whichcontains one or more synchronization codes. Specifically, when a frameis selected for synchronization insertion, and a scale factor LSB doesnot match (e.g. 0 where a 1 is indicated, or a 1 instead of a 0), thatscale factor is decremented and all the coefficients in the SFB areadjusted accordingly. Although the synchronization code can be damaged,random flipping of scale factor LSB by an attacker will introduceartifacts. To recover the watermark, a synchronization code is soughtand the data is recovered a manner appropriate to the watermarkingmethod.

To evaluate the audio watermarking system of FIG. 5, AT&T'simplementation of AAC was used. Watermark synchronization is indicatedby the sequence comprising the LSB of the first 44 decoded scale factorsin a long block. When the value of the LSB of a scale factor does notmatch the corresponding bit in the synchronization code then the scalefactor is decremented and the spectral coefficients adjustedaccordingly, resulting in perceptually irrelevant overcoding of theassociated spectral data.

The table of FIG. 6 shows the cost of carrying watermark data insertedby the embodiment of FIG. 5 into every frame of an AAC bitstream for astereo signal sampled at 44.1 kHz and coded at 96 kbps. Cost isexpressed as increase in bits per frame 610 (21.3 ms of audio) andincrease in rate 620, and was measured for both synchronization 630 andsynchronization+32 bits 640 cases. As can be seen in FIG. 6, theincrease in bits per marked frame 610 was 5.2 for synchronization 630and 9.0 for synchronization+32 640. The increase in rate 620 was 0.25%and 0.44%, respectively.

An important issue for any watermarking algorithm is the quality of thereconstructed signal following an attack which erases the watermark. Anaive attack on this marking algorithm has been simulated by zeroing allscale factor LSB. This attack results in unacceptable distortion in thereconstructed audio signal.

The baseline system for video compression uses a rudimentary perceptualmodel. A variance-based activity measure is used to select thequantization step-size for each macroblock, as in step 3 of the MPEG-2TM5 rate control [MPEG video committee, “Test Model 5”,ISO-IEC/JC1/SC29/WG11 N0400, Apr. 1993]. I frames are generated everyhalf second; all other frames are P frames. Watermark data was insertedinto both I and P frames, and the results were taken from an averageover two different 10 second sequences.

The first 44 macroblocks of a frame are used for synchronization. Thenext several macroblocks (100 or 600 in the Table, out of 1320) of aframe carry mark bits using the embodiment of FIG. 5. For eachmacroblock, when the LSB of the step-size Q_(p) does not match, Q_(p) isdecremented. However, a dead-zone is applied to the original Q_(p) toensure that zero coefficients remain zero.

A table showing the results of this simulation is shown in FIG. 7. Asseen in FIG. 7, the increase in bits per marked frame 710 was 124 forsynchronization 730, 138 for synchronization+100 bits 740, and 557 forsynchronization+600 bits 750. The corresponding increases in rate 720were 0.005%, 0.006%, and 0.024%, respectively. Simulation of a naiveattack on this algorithm by zeroing all scale factor LSBs demonstratesthat this attack results in a perceptible 1.6 dB degradation in PSNR ofthe reconstructed video signal.

What has been described is merely illustrative of the application of theprinciples of the present invention. Other arrangements, methods,modifications and substitutions by one of ordinary skill in the art arealso considered to be within the scope of the present invention, whichis not to be limited except by the claims which follow.

What is claimed is:
 1. A perceptual coder for encoding at least onecompressed audio or video signal to include hidden data, comprising, incombination: means for setting quantization step sizes to obtain a setof integer values after quantization of said at least one compressedsignal, said means for setting step sizes employing at least oneperceptual technique; means for adding the hidden data to said at leastone compressed signal; means for quantizing said at least one compressedsignal; and means for selecting a set of multipliers during thequantization, said adding of the hidden data occurring during thequantization.
 2. The coder of claim 1, further including means forselecting a specific Huffman Table for encoding said at least onecompressed signal.
 3. The coder of claim 2, wherein said selectedHuffman Table is not the Huffman Table that would normally be selectedfor encoding said at least one compressed signal.
 4. The coder of claim1, wherein said hidden data has a watermarking function.
 5. The coder ofclaim 4, wherein said hidden data has the form of at least one watermarkdata bit.
 6. The coder of claim 1, further including: means formodifying said encoded compressed signal by utilizing said set ofmultipliers.
 7. The coder of claim 6, wherein each member of said set ofmultipliers is close to unity.
 8. The coder of claim 7, wherein saidcompressed signal has at least one associated scale factor and saidmeans for modifying comprises, in combination: means for dividing saidat least one scale factor by a respective one of said multipliers; andmeans for multiplying said integer values by respective ones of saidmultipliers.
 9. The coder of claim 6, wherein said compressed signal hasat least one associated scale factor and said means for modifyingcomprises, in combination: means for dividing said at least one scalefactor by a respective one of said multipliers; and means formultiplying said integer values by respective ones of said multipliers.10. The coder of claim 1 wherein said means for adding hidden data isresponsive to said means for quantizing said at least one compressedsignal.
 11. A perceptual coder for encoding at least one of compressedaudio or video signal to include hidden data, comprising, incombination: means for setting quantization step sizes to obtain a setof integer values after quantization of said at least one compressedsignal, said means for setting step sizes employing at least oneperceptual technique; means for quantizing said at least one compressedsignal containing said hidden data; noiseless coding means for addingthe hidden data to said quantized compressed signals; and means forselecting a set of multipliers during the quantization, said adding ofthe hidden data occurring during the quantization.
 12. The coder ofclaim 11, wherein said hidden data has a watermarking function.
 13. Thecoder of claim 12, wherein said hidden data is added to at least onenon-zero one of said integer values.
 14. The coder of claim 11, furtherincluding: means for modifying said encoded compressed signal byutilizing said set of multipliers.
 15. The coder of claim 14, whereinsaid compressed signal has at least one associated scale factor and saidmeans for modifying comprises, in combination: means for dividing saidat least one scale factor by a respective one of said multipliers; andmeans for multiplying said integer values by respective ones of saidmultipliers.
 16. The coder of claim 14 wherein each member, x, of saidset of multipliers is equal to 2^(Ni), where N is a positive integer andi represents a set of indices associated with said compressed signal.17. The coder of claim 16, wherein said compressed signal has at leastone associated scale factor and said means for modifying comprises, incombination: means for dividing said at least one scale factor by arespective one of said multipliers; and means for multiplying saidinteger values by respective ones of said multipliers.
 18. A method ofperceptually encoding at least one compressed audio or video signals toinclude hidden data comprising, in combination, the steps of: settingquantization step sizes to obtain a set of integer values afterquantization of said at least one compressed signal, said setting ofstep sizes employing at least one perceptual technique; adding thehidden data to said at least one compressed signal; quantizing said atleast one compressed signal; and selecting a set of multipliers duringthe quantization, said adding of the hidden data occurring during thequantization.
 19. The method of claim 18, further including the step ofselecting a specific Huffman Table for encoding said at least onecompressed signal.
 20. The method of claim 19, wherein said selectedHuffman Table is not the Huffman Table that would normally be selectedfor encoding said at least one compressed signal.
 21. The method ofclaim 18, wherein said hidden data has a watermarking function.
 22. Themethod of claim 21, wherein said hidden data has the form of at leastone watermark data bit.
 23. The method of claim 18, further including:modifying said encoded compressed signal by utilizing said set ofmultipliers.
 24. The method of claim 23, wherein each member of said setof multipliers is close to unity.
 25. The method of claim 24, whereinsaid compressed signal has at least one associated scale factor and stepof modifying comprises the steps, in combination, of: dividing said atleast one scale factor by a respective one of said multipliers; andmultiplying said integer values by respective ones of said multipliers.26. The method of claim 23, wherein said compressed signal has at leastone associated scale factor and said step of modifying comprises thesteps, in combination, of: dividing said at least one scale factor by arespective one of said multipliers; and multiplying said integer valuesby respective ones of said multipliers.
 27. The method of claim 18,further including the step of marking said at least one audio or videosignal with hidden data before compression.
 28. The method of claim 18,further including the step of bitstream marking said quantizedcompressed signal with hidden data.
 29. The method of claim 28, furtherincluding the step of marking said at least one audio or video signalwith hidden data before compression.
 30. The method of claim 18 whereinsaid step of adding hidden data occurs simultaneously with said step ofquantizing said at least one compressed signal.
 31. The method of claim18 wherein said step of adding hidden data occurs after said step ofquantizing said at least one compressed signal.
 32. A method ofperceptually encoding at least one compressed audio or video signal toinclude hidden data comprising, in combination, the steps of: settingquantization step sizes to obtain a set of integer values afterquantization of said at least one compressed signal, said setting ofstep sizes employing at least one perceptual technique; quantizing saidat least one compressed signal containing said hidden data; adding thehidden data to said quantized compressed signal; and selecting a set ofmultipliers during the quantization, said adding of the hidden dataoccurring during quantization.
 33. The method of claim 32, wherein saidhidden data has a watermarking function.
 34. The method of claim 33,wherein said hidden data is added to at least one non-zero one of saidinteger values.
 35. The method of claim 32, further including: modifyingsaid encoded compressed signal by utilizing said set of multipliers. 36.The coder of claim 35, wherein said compressed signal has at least oneassociated scale factor and said step of modifying comprises, incombination: dividing said at least one scale factor by a respective oneof said multipliers; and multiplying said integer values by respectiveones of said multipliers.
 37. The method of claim 35, wherein eachmember, x, of said set of multipliers is equal to 2^(Ni), where N is apositive integer and i represents a set of indices associated with saidcompressed signal.
 38. The method of claim 37, wherein said compressedsignal has at least one associated scale factor and said step ofmodifying comprises, in combination: dividing said at least one scalefactor by a respective one of said multipliers; and multiplying saidinteger values by respective ones of said multipliers.
 39. The method ofclaim 32, further including the step of marking said at least one audioor video signal with hidden data before compression.
 40. The method ofclaim 32, further including the step of bitstream marking said quantizedcompressed signal with hidden data.
 41. The method of claim 40, furtherincluding the step of marking said at least one audio or video signalwith hidden data before compression.
 42. A method of perceptually codingas signal to add watermark data using a codebook characterized by thesteps of: establishing scale factors from perceptual thresholds;establishing scale factor bands to watermark; choosing a set ofmultipliers approximately equal to 1; and dividing the scale factors byrespective multipliers of said set of multipliers for selecting saidcodebook.